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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

IThis document has been translated by connputer. So the translation nnay not reflect the 
original precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the lnvention]This invention relates to the parallax image input device and imaging 

device which capture the image seen from two or more viewpoints. 

[0002] 

[Description of the Prior Art]The art of putting a camera on several different viewpoints 
conventionally, extracting the corresponding points between two or more pictures which 
captured and captured the image of the same photographic subject in each viewpoint, asking 
for the distance map to a photographic subject, and asking for the three-dimensional shape of 
a photographic subject based on this distance map is known. Thus, the device which photos a 
parallax picture is known as a device which captures an image from a different viewpoint, 
moving the whole photographing system. For example, the camera moving mechanism to 
which the whole camera is moved is indicated by JP,9-44677,A (it is called the literature 1). 
[0003]To JP,10-229566,A (it is called the literature 2). The digital camera system having a 
means to carry out the seriography of the same photographic subject, and to save two or more 
pictures while moving, and a means to extract the movement magnitude of the image element 
between two or more saved pictures as depth information of a photographic subject is 
indicated. In order to photo a parallax picture, a camera is changed to animation 
photographing mode, and this system is indicated to carry out the seriography of the same 
photographic subject, moving the camera concerned in the direction which goes to an optical 
axis direction when a still picture is photoed direct. 

[0004]On the other hand, to JP,1 0-4231 4,A (it is called the literature 3). The light transmission 
range of the light control means which defines the light transmission range in the pupil surface 
of an optical lens, and a light control means is moved in the direction parallel to the pupil 
surface of said optical lens, The light transmission range control means of the light control 
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means which transmits selectively only the picture information which passes the specific 
position of an optical lens, The parallax image input device of the single lens method 
consisting of an imaging means which picturizes the picture information transmitted by the light 
control means, and is changed into an image data sequence is indicated. 
[0005]To JP,1 0-271 534,A (it is called the literature 4). It is arranged at the position from which 
only a predetermined distance separated the light of the photographic subject focusing on the 
optic-axis middle of the optical system which carries out image formation to a solid state image 
sensor, and an optical system, have two or more light modulation mechanisms which 
penetrate or shade the light from a photographic subject, and with a predetermined time 
interval. One of said each of the light modulation mechanism is made into a transmission state, 
and the stereoscopic picture photographing instrument which makes the remainder a shaded 
state is indicated. 
[0006] 

[Problem(s) to be Solved by the Inventionjin the invention indicated in the above-mentioned 
literature 1, there was a problem that a camera moving mechanism and a control device 
became large-scale. Change a camera to animation photographing mode, in the invention 
indicated in the literature 2, in order to photo a parallax picture, there is a statement that the 
seriography of the same photographic subject is carried out, moving in the direction which 
goes to an optical axis direction when a still picture is photoed direct, but. It is not indicated 
how a camera is moved. Although it is thought that the moving mechanism which can be 
controlled with high precision in this case, for example, a means to measure the locus of 
camera movement in three dimensions, and to record it, the mechanism in which the locus of 
movement of a camera is presumed, etc. are needed, it is dramatically difficult to prepare such 
a device, and it is not realistic. 

[0007]ln the device indicated to the literature 3 or 4, the angle variation of the optic axis is 
included in the picture acquired with azimuth difference by the effect of the refracting power of 
a lens, the operation which computes the parallax amount about a predetermined photographic 
subject and the depth information (depth position) of the photographic subject concerned from 
a picture became complicated, and the problem that processing time was long had arisen. 
Then, an object of this invention is to provide the parallax image input device and imaging 
device which can obtain a parallax picture easily and can acquire the depth information of a 
photographic subject easily. This purpose is attained by the combination of the feature given in 
the independent paragraph in a claim. A dependent claim specifies the further advantageous 
example of this invention. 
[0008] 

[Means for Solving the ProblemjTo achieve the above objects, a parallax image input device 
which this invention requires for the 1st gestalt of this invention is characterized by that a 
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parallax image input device which acquires a picture of the external world seen fronn two or 
more viewpoints comprises the following. 

An image formation part for azimuth difference which connects a picture of the external world 
seen from a viewpoint. 

An image pick-up part for azimuth difference which captures an image of the external world 

seen from a viewpoint connected by image formation part for azimuth difference. 

An image formation part actuator for azimuth difference which moves an image formation part 

for azimuth difference, and changes a viewpoint to an image pick-up part for azimuth 

difference. 

[0009]A parallax image input device which this invention applies to the 2nd gestalt of this 
invention to achieve the above objects is characterized by that a parallax image input device 
which acquires a picture of the external world seen from two or more viewpoints comprises: 
An image formation part for azimuth difference which connects a picture of the external world 
seen from a viewpoint. 
Case. 

An image formation part actuator for azimuth difference which is held at a case, moves an 

image formation part for azimuth difference, and changes a viewpoint. 

An image pick-up part for azimuth difference which captures an image of two or more external 

worlds seen from a different viewpoint connected by image formation part for azimuth 

difference. 

[0010]lt has further an image pick-up part actuator for azimuth difference which moves an 
image pick-up part for azimuth difference to a case, and may be made to change a position of 
an image pick-up part for azimuth difference for every viewpoint by an image pick-up part 
actuator for azimuth difference. It may be made for an image pick-up part actuator for azimuth 
difference to move an image pick-up part for azimuth difference based on an interval of 
movement of parallax directions of an image formation part for azimuth difference by an image 
formation part actuator for azimuth difference. It may be made for a taking-in control section to 
be provided with a taking-in control section which changes a range which captures an image 
on an acceptance surface of an image pick-up part for azimuth difference for every viewpoint. 
[0011]To achieve the above objects, a parallax image input device which this invention 
requires for the 3rd gestalt of this invention is characterized by that a parallax image input 
device which acquires a picture of the external world seen from two or more viewpoints 
comprises the following. 

An image formation part for azimuth difference of a single optical axis which connects a picture 

of the external world. 
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A light passage control section which has the 1st opening and the 2nd opening used as a 
viewpoint which passes light which penetrated an innage fornnation part for azinnuth difference. 
An image pick-up part for azinnuth difference which captures the 1st picture of the external 
world by light which passed the 1st opening of a light passage control section, and the 2nd 
image of the external world by light which passed the 2nd opening. 

A taking-in control section which changes a range which captures an image on an acceptance 
surface of an image pick-up part for azimuth difference based on an interval with the 1st 
opening and the 2nd opening by picture seen from the 1st opening, and a picture seen from 
the 2nd opening. 

It may be made to have further a depth primary detecting element which detects depth 
information of a predetermined photographic subject of the external world in a picture input 
device for azimuth difference concerning this invention based on a picture captured by image 
pick-up part for azimuth difference. 

[0012]To achieve the above objects, an imaging device which this invention requires for the 1st 
gestalt of this invention is characterized by that an imaging device which picturizes the desired 
external world comprises the following. 

An image formation system which connects a picture of the external world seen from two or 
more viewpoints. 

An imaging system which picturizes a picture connected by an image formation system. 

An image formation system actuator which moves an image formation system and changes a 

viewpoint to an imaging system. 

A control section which controls an image formation system or an imaging system based on 
depth information detected by a depth primary detecting element which detects depth 
information corresponding to distance to a predetermined photographic subject of the external 
world, and depth primary detecting element based on a picture of the external world seen from 
a different viewpoint picturized by imaging system. 

[0013]Have an image formation system and an image formation part which connects a picture 
of the external world from a viewpoint, and an image formation part for azimuth difference 
which connects a picture of the external world seen from two or more viewpoints an imaging 
system. Have an image pick-up part which picturizes a picture connected by image formation 
part, and an image pick-up part for azimuth difference which captures an image of the external 
world seen from a viewpoint connected by image formation part for azimuth difference, and an 
image formation system actuator. Have an image formation part actuator for azimuth 
difference which moves an image formation part for azimuth difference, and changes a 
viewpoint to an image pick-up part for azimuth difference, and a depth primary detecting 
element. Based on a picture of the external world seen from a different viewpoint picturized by 
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image pick-up part for azimuth difference, deptli information corresponding to distance to a 
predetermined pliotographic subject of the external world is detected, and it may be made for a 
control section to control an image formation part or an image pick-up part based on depth 
information detected by depth primary detecting element. 

[0014]To achieve the above objects, an imaging device which this invention requires for the 
2nd gestalt of this invention is characterized by that an imaging device which picturizes the 
desired external world comprises the following. 

An image formation system which connects a picture of the external world seen from two or 
more viewpoints. 

An imaging system which picturizes a picture connected by image formation part. 
Case. 

An image formation system actuator which moves an image formation system and changes a 
viewpoint to a case, A control section which controls an image formation system or an imaging 
system based on depth information detected by a depth primary detecting element which 
detects depth information corresponding to distance to a predetermined photographic subject 
of the external world, and depth primary detecting element based on a picture of the external 
world seen from a different viewpoint picturized by imaging system. 

[0015]Have an image formation system and an image formation part which connects a picture 
of the external world, and an image formation part for azimuth difference which connects a 
picture of the external world seen from a viewpoint an imaging system. Have an image pick-up 
part which picturizes a picture connected by image formation part, and an image pick-up part 
for azimuth difference which captures an image of two or more external worlds seen from a 
different viewpoint connected by image formation part for azimuth difference, and an image 
formation system actuator, Have an image formation part actuator for azimuth difference which 
moves an image formation part for azimuth difference, and changes a viewpoint to a case, and 
a depth primary detecting element, Based on a picture of the external world seen from a 
different viewpoint picturized by image pick-up part for azimuth difference, depth information 
corresponding to distance to a predetermined photographic subject of the external world is 
detected, and it may be made for a control section to control an image formation part or an 
image pick-up part based on depth information detected by depth primary detecting element. 
[0016]An imaging device which this invention applies to the 3rd gestalt of this invention to 
achieve the above objects is characterized by that an imaging device which picturizes the 
desired external world comprises: 

An image formation system which connects a picture of the external world. 

A light passage control section which has the 1st opening and the 2nd opening used as a 

viewpoint which passes light which penetrated an image formation system. 
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An imaging systenn wliicli picturizes a picture connected by an innage fornnation system. 
Based on an interval with the 1st opening and the 2nd opening, by picture seen from the 1st 
opening, and a picture seen from the 2nd opening. A taking-in control section which changes a 
range which captures an image on an acceptance surface of an imaging system, A control 
section by which a range incorporated by a taking-in control section controls an image 
formation system or an imaging system based on depth information detected by a depth 
primary detecting element which detects depth information corresponding to distance to a 
predetermined photographic subject of the external world, and depth primary detecting 
element based on a different-****(ed) picture. 

[0017]Have an image formation system and an image formation part which connects a picture 
of the external world, and an image formation part for azimuth difference of a single optical 
axis which connects a picture of the external world seen from a viewpoint a light passage 
control section, Have the 1st opening and the 2nd opening used as a viewpoint which passes 
light which penetrated an image formation part for azimuth difference, and an imaging system. 
It has an image pick-up part for azimuth difference which captures the 1st picture of the 
external world by light which passed the 1st opening of an image pick-up part which picturizes 
a picture connected by image formation part, and a light passage control section, and the 2nd 
image of the external world by light which passed the 2nd opening, A taking-in control section 
is with a picture seen from the 1st opening based on an interval with the 1st opening and the 
2nd opening, and a picture seen from the 2nd opening, Make ranges which capture an image 
on an acceptance surface of an image pick-up part for azimuth difference differ them, and a 
depth primary detecting element. Based on a picture captured by image pick-up part for 
azimuth difference, depth information corresponding to distance to a predetermined 
photographic subject of the external world is detected, and it may be made for a control section 
to control an image formation part or an image pick-up part based on depth information 
detected by depth primary detecting element. 

[OOlSjIt may be made for an image pick-up part to have an installation section which it may be 
made to have an optoelectric transducer and installs a photochemical reaction member. An 
outline of the above-mentioned invention is not what enumerated all the required features of 
this invention, and a subcombination of these characterizing group can also be invented. 
[0019] 

[Embodiment of the InventionjAlthough this invention is hereafter explained through an 
embodiment of the invention, not all the combination of the feature of following embodiments 
that do not limit the invention concerning a claim and are explained in the embodiment is 
necessarily indispensable to the solving means of an invention. Dravying 1 shows the 
composition of the digital camera as an example of the imaging device containing the parallax 
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image input device concerning a 1st ennbodinnent of tliis invention. Here, tlie video camera etc. 
wliicli picturize continuously not only the camera that picturizes a picture for every sheet but a 
picture are contained in a digital camera. The imaging device 10 is provided with the following. 
It has the case 1 1 and is the image formation part 24 in the case 1 1 concerned. 
Image pick-up part 26. 
Storage parts store 28. 

The parallax image input device body part 12 which has the picture slicing part 17, the parallax 
picture storage parts store 18, the depth primary detecting element 20, and the depth storage 
parts store 22 as an example of the actuator 15 as an example of the image formation part 14 
for azimuth difference, an image formation system actuator, and the image formation part 
actuator for azimuth difference, the image pick-up part 16 for azimuth difference, and a taking- 
in control section, and the control section 30. 

Here, the image formation system said to a claim is constituted by the image formation part 24 
and the image formation part 14 for azimuth difference, and an imaging system is constituted 
from this embodiment by the image pick-up part 26 and the image pick-up part 16 for azimuth 
difference. 

[0020]The image formation part 24 has the singular number or two or more lenses, and 
connects the picture of the external world on the acceptance surface of the image pick-up part 
26 by collecting the lights from the external world, for example. The image pick-up part 26 
captures the image connected on the acceptance surface. According to this embodiment, the 
image pick-up part 26 is CCD (Charge Coupled Device) as an example of an optoelectric 
transducer, changes into image data the picture connected on the acceptance surface, and 
captures it. The storage parts store 28 memorizes the image data changed by the image pick- 
up part 26. 

[0021]The image formation part 14 for azimuth difference has the singular number or two or 
more lenses, and connects the picture of the external world on the acceptance surface of the 
image pick-up part 16 for azimuth difference, for example. The actuator 15 moves the image 
formation part 14 for azimuth difference to the case 1 1. Thereby, the picture seen from two or 
more viewpoints can be acquired. In this embodiment, it moves the image formation part 14 for 
azimuth difference in the direction vertical to the optic axis concerned, the actuator 15 
maintaining an optic axis in parallel. It is made to move in this embodiment between two view 
positions (one side is called 1st view position and another side is called 2nd view position) 
which were able to determine beforehand the image formation part 14 for azimuth difference 
by the actuator 15. If it does in this way, composition of the actuator 15 can be simplified, and 
the interval of the parallax directions to which a view position is connected can always be fixed, 
and it is not necessary to have the composition which measures the interval of the parallax 
directions of a view position. 



http://www4.ipdl.inpit.go.ip/cgi-bin/tran_web_cgi_eiie?atw_u=http%3A%2F%2Fwww4.i... 12/20/2008 



JP,2001-012927,A [DETAILED DESCRIPTION] 



Page 8 of 18 



[0022]The image pick-up part 16 for azimuth difference captures the image of the external 
world seen from two or more viewpoints connected by the image formation part 14 for azimuth 
difference. According to this embodiment, the image pick-up part 16 for azimuth difference is 
CCD as an example of an optoelectric transducer, changes into image data the picture 
connected on the acceptance surface, and captures it. In this embodiment, it is fixed to the 
case 1 1 and the image pick-up 16 for azimuth difference cannot be moved. The picture slicing 
part 17 determines a logging (cropping) range which is different based on the interval of the 
parallax directions to which the viewpoint about two or more viewpoints is connected, 
respectively about two or more image data seen from two or more viewpoints, and starts each 
image data according to the range concerned. 

[0023]lt may be made to change here the logging range of two or more Image data seen from 
two or more viewpoints depended on the picture slicing part 17 by the value obtained by the 
linearity predetermined function which makes these viewpoint Intervals a variable. If It does In 
this way, the image data from two or more viewpoints acquired by making move the whole 
camera as shown In a conventional example in parallel with the optic axis of a camera lens, 
and the same Image data can be obtained. Therefore, it can ask for the depth position of the 
predetermined photographic subject in a picture easily by performing the same processing as 
a conventional example from the obtained image data. 

[0024]lt may be made to change the logging range of two or more Image data seen from two or 
more viewpoints depended on the picture slicing part 17 by the value obtained by the nonlinear 
predetermined function which makes a viewpoint interval a variable. If It does In this way, the 
optic axis of a camera lens can be rotated for the whole camera focusing on a predetermined 
point, and the image data from two or more viewpoints further acquired by moving the whole 
camera In parallel with the optic axis of a camera lens and the same image data can be 
obtained. 

[0025]ln this embodiment, the picture slicing part 17 about Image data In case the Image 
formation part 14 for azimuth difference Is In the 1st view position. The range of the 
predetermined width centering on the position of the acceptance surface of the Image pick-up 
part 1 6 for azimuth difference equivalent to the optic axis (optic axis 1 ) of the Image formation 
part 14 for azimuth difference concerned Is started, About image data In case the image 
formation part 14 for azimuth difference Is in the 2nd view position. The range which shifted the 
logging range In case the said range 14 of predetermined width centering on the position of the 
acceptance surface of the image pick-up part 16 for azimuth difference equivalent to the optic 
axis (optic axis 2) of the image pick-up part 1 for azimuth difference concerned, i.e., said image 
formation part for azimuth difference, is in the 1st view position by the interval of the 1st and 
2nd view positions is started. This is an example at the time of changing the above-mentioned 
logging range by the value obtained by a linearity function. 
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[0026]The picture slicing part 17 as a method of starting the image data of the predetermined 
range from image data, It may be made to start the image data of the predetermined range 
from the image data of the analog signal changed into the image pick-up part 16 for azimuth 
difference, After the image pick-up part 16 for azimuth difference changes the image data of an 
analog signal into the image data of a digital signal, it may be made to start the image data 
which shows the predetermined range from the image data concerned. 
[0027]The parallax picture storage parts store 18 memorizes the image data of the external 
world seen from two or more viewpoints started by the picture slicing part 17. The depth 
primary detecting element 20 shifts based on the image data seen from a different viewpoint 
memorized by the parallax picture storage parts store 18 by performing corresponding-points 
decision processing about the predetermined photographic subject of the external world, and 
calculates quantity. Since corresponding-points decision processing is art known 
conventionally, it omits explanation. Corresponding-points decision processing is indicated to 
the "Chapter 8 stereo **" etc. of "computer vision:technical comment, a future view. New 
technology Communications, 1998, ISBN4-915851-17-6." 

[0028]The depth primary detecting element 20 performs depth position detection processing 
which detects the depth position (depth information) of a photographic subject based on the 
amount of gaps concerned. The function which shows the relation between the amount of gaps 
and a depth position is searched for beforehand, and it may be made to compute a depth 
position by shifting to the function concerned and substituting quantity in depth position 
detection processing. The correspondence relation between the amount of gaps and a depth 
position is held as a table, and it shifts from the table concerned and may be made to take out 
the depth position corresponding to quantity. Since the relation between the amount of gaps 
and a depth position can be expressed based on the principle of triangulation known 
conventionally, the character of a lens, the principle of geometry, etc., for example, it omits 
explanation here. 

[0029]The depth storage parts store 22 memorizes the depth position of the photographic 
subject detected by the depth primary detecting element 20. The control section 30 controls 
the focus of the image formation part 24, the imaging operation by the image pick-up part 26, 
etc. based on the depth position of the predetermined photographic subject memorized by the 
depth storage parts store 22. Here the storage parts store 28, the parallax picture storage 
parts store 18, and the depth storage parts store 22, RAM established permanently in the 
imaging device 10, respectively (Random Access Memory), To the imaging device 10, it may 
be a flash memory and they may be [ it is removable, for example ] recording media, such as a 
floppy disk, MD (Mini Disk), and SmartMedia (trademark). 

[OOSOjDravying 2 is a figure explaining the photographing operation of the picture by the 
imaging device concerning a 1st embodiment of this invention. Drawing 2 (A) is a figure 
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explaining pliotograpliy in case the image formation part 14 for azimuth difference is in the 1st 
view position, and drawing 2 (B) is a figure explaining photography in case the image formation 
part 14 for azimuth difference is in the 2nd view position. Here, the X-axis shall be taken in 
parallel with the move direction of the image formation part 14 for azimuth difference. The 
photographic subject A whose distance parallel to an optic axis from the image formation part 
14 for azimuth difference in the external world is the distance A. There is the photographic 
subject C whose distance parallel to an optic axis is infinity (infinity) from the photographic 
subject B whose distance parallel to an optic axis from the image formation part 14 for azimuth 
difference is the distance B, and the image formation part 14 for azimuth difference, and it is 
assumed that the photographic subject A, B, and C is located in a line sequentially from the 
one where the value of the X-axis in a figure is larger. 

[0031 ]The actuator 15 makes the 1st view position maintain the image formation part 14 for 
azimuth difference first in the imaging device 10. The optic axis of the image formation part 14 
for azimuth difference in the 1st view position is used as the optic axis 1. Thereby, in the 1st 
view position, the picture of the external world is connected via the image formation part 14 for 
azimuth difference on the acceptance surface of the image pick-up part 16 for azimuth 
difference. As shown in drawing 2 (A), the image of each photographic subject is connected to 
the image pick-up part 16 for azimuth difference in order of the photographic subject C, B, and 
A sequentially from the one where the value of the X-axis is larger, for example. The image 
pick-up part 16 for azimuth difference changes into image data the picture connected on the 
acceptance surface. 

[0032]Subsequently, it is made to move to the 2nd view position, the actuator 15 maintaining 
the optic axis of the image formation part 14 for azimuth difference in parallel with the optic 
axis 1 . Here, the optic axis of the image formation part 14 for azimuth difference in the 2nd 
view position is used as the optic axis 2. Thereby, in the 2nd view position, the picture of the 
external world is connected via the image formation part 14 for azimuth difference on the 
acceptance surface of the image pick-up part 16 for azimuth difference. As shown in drawing 2 
(B), the image of each photographic subject is connected to the image pick-up part 16 for 
azimuth difference in order of the photographic subject C, B, and A sequentially from the one 
where the value of the X-axis is larger. As compared with the position of the image shown in 
drawing 2 (A), all the photographic subjects A, B, and C are connected to the position shifted in 
the direction which becomes small, the value of the X-axis shifts in order of the photographic 
subject C, B, and A, and the position of the image shown in drawing 2 (B) has much quantity. 
The image pick-up part 16 for azimuth difference changes into image data the picture 
connected on the acceptance surface. 

[0033]Dravying 3 is a figure explaining logging operation of the picture concerning a 1st 
embodiment of this invention. Drawing 3 is a figure explaining logging operation of a picture as 
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shown in drawing 2 , when a picture is connected in the 1st view position and the 2nd view 
position. The picture slicing part 17 starts the 1st innage data shown in drawing 3 (C) contained 
in predetermined width focusing on the position of the acceptance surface of the innage 
fornnation part 16 for azimuth difference equivalent to the optic axis 1 to the image connected 
to the image formation part 16 for azimuth difference in the 1st view position as shown in 
drawing 3 (A). The picture slicing part 17 starts the 2nd image data shown in drawing 3 (D) 
contained in predetermined width focusing on the position of the acceptance surface of the 
image formation part 16 for azimuth difference equivalent to the optic axis 2 to the image 
connected to the image formation part 16 for azimuth difference in the 2nd view position as 
shown in drawing 3 (B). 

[0034]lf the 1st image data started in this way and the 2nd image data are piled up, the amount 
of gaps of the same photographic subject between both-images data will appear greatly, so 
that the same photographic subject C with an infinite distance is in agreement and the distance 
of a photographic subject becomes near, as shown in drawing 3 (E). As opposed to the image 
data which the image data shown in drawing 3 (E) is the image data from two or more 
viewpoints acquired by making move the whole camera shown in a conventional example in 
parallel with the optic axis of a camera lens, and same image data, and was obtained. It can 
ask for the depth position of the predetermined photographic subject in a picture easily by 
performing the same processing as a conventional example. These image data is memorized 
by the parallax picture storage parts store 18. 

[0035]Next, the processing after making the image data in the imaging device 10 memorize is 
explained. Based on the image data which was started by the picture slicing part 17 and 
memorized by the parallax picture storage parts store 18 as mentioned above in the imaging 
device 10, The depth position of the photographic subject concerned which the depth primary 
detecting element 20 shifted about the predetermined photographic subject of the external 
world by corresponding-points decision processing, detected quantity, and detected the depth 
position about the photographic subject concerned based on the amount of gaps concerned 
and in which the depth storage parts store 22 was detected is memorized. 
[0036]Subsequently, the control section 30 controls the image formation part 24 and the image 
pick-up part 26 based on the depth position of the photographic subject memorized by the 
depth storage parts store 22. Thereby, the image formation part 24 collects the lights from the 
external world, and connects the picture of the photographic subject of the external world on 
the acceptance surface of the image pick-up part 26. And the image pick-up part 26 changes 
into image data the picture connected on the self acceptance surface, and the storage parts 
store 28 memorizes the image data changed by the image pick-up part 26. 
[0037]At this time, the storage parts store 28 is faced incorporating the image data which self 
memorizes, and the image data concerned, The matching information which matches the 



http://www4.ipdl.inpit.go.ip/cgi-bin/tran_web_cgi_eiie?atw_u=http%3A%2F%2Fwww4.i... 12/20/2008 



JP,2001-012927,A [DETAILED DESCRIPTION] 



Page 12 of 18 



depth position of tlie pliotograpliic subject which is computed by the picture from two or more 
viewpoints which were incorporated by the image pick-up part 16 for azimuth difference, and 
were memorized by the parallax picture storage parts store 18, and the picture from two or 
more viewpoints concerned, and is memorized by the depth storage parts store 22 is also 
memorized. Thereby, image data, the picture from two or more viewpoints related with the 
image data concerned, and the depth information of the photographic subject included in the 
image data concerned can be matched and used behind. 

[0038]As mentioned above, only by moving only the image formation part 14 for azimuth 
difference, without moving the imaging device 10 whole. Since the image of the external world 
seen from two or more viewpoints can be captured, it is not necessary to have a drive system 
which moves the imaging device 10 whole, and things can be carried out and the cost 
concerning an imaging device and a parallax image input device which miniaturizes the size of 
an imaging device and a parallax image input device can be reduced. It can ask for the depth 
position of the predetermined photographic subject in a picture easily. 
[0039]Drawing 4 is a figure showing the composition of the imaging device containing the 
parallax image input device concerning a 2nd embodiment of this invention. Here, identical 
codes are given to the element which has the same function as the element of the imaging 
device of a 1st embodiment shown in .dravying 1. In the imaging device 10 of a 1st 
embodiment, this imaging device 10 is changed into the actuator 15, is provided with the 
actuator 32 as an example of the image pick-up part actuator for azimuth difference, and has 
composition which removed the picture slicing part 17. Here, the image formation system said 
to a claim is constituted by the image formation part 24 and the image formation part 14 for 
azimuth difference, and an imaging system is constituted from this embodiment by the image 
pick-up part 26 and the image pick-up part 16 for azimuth difference. 

[0040]the actuator 32 ~ the function of the actuator 1 5 of a 1 st embodiment ~ in addition, it has 
further the function to make the image pick-up part 16 for azimuth difference drive. The 
actuator 32 changes the position of the image pick-up part 16 for azimuth difference which 
captures the image seen from each viewpoint based on the interval of the parallax directions to 
which the viewpoint of two or more viewpoints is connected. It may be made to change the 
actuator 32 here by the value obtained [ interval / of the parallax directions of these 
viewpoints ] by the linearity predetermined function made into a variable in the position of each 
image pick-up part 16 for azimuth difference at the time of incorporating each image data seen 
from two or more viewpoints. If it does in this way, the image data from two or more viewpoints 
acquired by making move the whole camera as shown in a conventional example in parallel 
with the optic axis of a camera lens, and the same image data can be obtained. Therefore, it 
can ask for the depth position of the predetermined photographic subject in a picture easily by 
performing the same processing as a conventional example from the obtained image data. 
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[0041] It may be made to change the actuator 32 by the value obtained [ intervals / these / 
viewpoint ] by the nonlinear predetermined function made into a variable In the position of each 
image picl<-up part 16 for azimuth difference at the time of incorporating each image data seen 
from two or more viewpoints. If it does in this way, the optic axis of a camera lens can be 
rotated for the whole camera focusing on a predetermined point, and the image data from two 
or more viewpoints further acquired by moving the whole camera in parallel with the optic axis 
of a camera lens and the same image data can be obtained. 

[0042]ln this embodiment, when the image formation part 14 for azimuth difference is in the 1st 
view position, the actuator 32, So that the center of the X axial direction of the acceptance 
surface of the image pick-up part 16 for azimuth difference and the optic axis (optic axis 1) of 
the image formation part 14 for azimuth difference concerned may be in agreement, When the 
image pick-up part 16 for azimuth difference is moved and the image formation part 14 for 
azimuth difference is in the 2nd view position, the image pick-up part 16 for azimuth difference 
is moved so that the center of the X axial direction of the acceptance surface of the Image 
pick-up part 16 for azimuth difference and the optic axis (optic axis 2) of the image formation 
part 14 for azimuth difference concerned may be in agreement. This is an example at the time 
of changing the above-mentioned logging range by the value obtained by a linearity function. 
[0043]Draw!ng 5 is a figure explaining the photographing operation of the parallax picture by 
the imaging device concerning a 2nd embodiment of this invention. Drawing^^^^ is a figure 
explaining photography in case the image formation part 14 for azimuth difference is in the 1st 
view position, and drawing 5 (B) is a figure explaining photography in case the image formation 
part 14 for azimuth difference is in the 2nd view position. The photographic subject A whose 
distance parallel to an optic axis from the image formation part 14 for azimuth difference in the 
external world is the distance A here. There is the photographic subject C whose distance 
parallel to an optic axis is infinity (infinity) from the photographic subject B whose distance 
parallel to an optic axis from the image formation part 14 for azimuth difference is the distance 
B, and the image formation part 14 for azimuth difference, and it is assumed that the 
photographic subject A, B, and C is located in a line sequentially from the one where the value 
of the X-axis in a figure is larger. 

[0044]ln the imaging device 10, first, while the actuator 32 makes the 1st view position 
maintain the image formation part 14 for azimuth difference, the image pick-up part 16 for 
azimuth difference is maintained so that the center of the optic axis (optic axis 1) of the image 
formation part 14 for azimuth difference concerned and the X-axis of an acceptance surface in 
the 1st view position may be in agreement. Thereby, in the 1st view position, the picture of the 
external world is connected via the image formation part 14 for azimuth difference on the 
acceptance surface of the image pick-up part 16 for azimuth difference. As shown in dravving 5 
(A), the image of each photographic subject is connected to the image pick-up part 16 for 
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azimuth difference in order of tlie photograpliic subject C, B, and A sequentially from the one 
where the value of the X-axis is larger, for example. The image pick-up part 16 for azimuth 
difference changes into image data the picture connected on the acceptance surface. 
[0045]Subsequently, while making it move to the 2nd view position, the actuator 32 
maintaining the optic axis of the image formation part 14 for azimuth difference in parallel with 
the optic axis 1 , the image pick-up part 16 for azimuth difference is moved so that the center of 
the optic axis (optic axis 2) of the image formation part 14 for azimuth difference concerned 
and the X-axis of an acceptance surface in the 2nd view position may be in agreement. 
Thereby, in the 2nd view position, the picture of the external world is connected via the image 
formation part 14 for azimuth difference on the acceptance surface of the image pick-up part 
16 for azimuth difference. As shown in draMQQA (B), the image of each photographic subject 
is connected to the image pick-up part 16 for azimuth difference in order of the photographic 
subject C, B, and A sequentially from the one where the value of the X-axis is larger. The 
image pick-up part 16 for azimuth difference changes into image data the picture connected on 
the acceptance surface. 

[0046]Thus, if the 1st image data changed into the image pick-up part 16 for azimuth 
difference and the 2nd image data are piled up, as shown in drawing 5 (C), the same 
photographic subject C with an infinite distance is in agreement, and the amount of gaps of the 
same photographic subject between both-images data will appear greatly, so that the distance 
of a photographic subject is near. As opposed to the image data which the image data shown 
in drawing 5 (C) is the image data from two or more viewpoints acquired by making move the 
whole camera shown in a conventional example in parallel with the optic axis of a camera lens, 
and same image data, and was obtained. It can ask for the depth position of the predetermined 
photographic subject in a picture easily by performing the same processing as a conventional 
example. This 1st image data and 2nd image data are memorized by the parallax picture 
storage parts store 18. About the processing after making the image data in the imaging 
device 10 memorize, since it is the same as that of a 1st above-mentioned embodiment, 
explanation is omitted here. 

[0047]Thus, only by moving the image formation part 14 for azimuth difference, and the image 
pick-up part 16 for azimuth difference, without moving the imaging device 10 whole. Since the 
image from two or more viewpoints can be captured, it is not necessary to have a device of the 
drive system which moves the imaging device 10 whole, and things can be carried out and the 
cost concerning an imaging device and a parallax image input device which miniaturizes the 
size of an imaging device and a parallax image input device can be reduced. It is not 
necessary to give a margin to the acceptance surface of the image pick-up part 16 for azimuth 
difference, the image pick-up part 16 for azimuth difference can be miniaturized, and cost can 
be reduced. It can ask for the depth position of the predetermined photographic subject in a 
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picture easily. 

[0048]Next, the imaging device concerning a 3rd embodinnent of tliis invention is explained. 
Drawing 6 is a figure showing the connposition of the parallax image input device of the 
imaging device concerning a 3rd embodiment of this invention. The imaging device concerning 
a 3rd embodiment changes the composition of the parallax image input device body part 12 of 
the imaging device concerning a 1st embodiment shown in drawing 1 . Here, about the same 
functional element as the composition in the parallax image input device body part 12 shown in 
drawing 1 , the same numerals are attached and the overlapping explanation is omitted. This 
parallax image input device body part 12 is provided with the following. 
The image formation part 14 for azimuth difference. 
Shutter part 50. 

The image pick-up part 16 for azimuth difference. 

The picture slicing part 17, the parallax picture storage parts store 18, the depth primary 
detecting element 20, the depth storage parts store 22, and the control section 52. 
Here, the image formation system said to a claim is constituted by the image formation part 24 
and the image formation part 14 for azimuth difference, and an imaging system is constituted 
from this embodiment by the image pick-up part 26 and the image pick-up part 16 for azimuth 
difference. The shutter part 50 has the opens parts 50A and 50B used as a viewpoint which 
can be opened and closed. As for the shutter part 50, it is preferred to be arranged in the pupil 
surface of the image formation part 14 for azimuth difference or its neighborhood. The control 
section 52 controls each part. For example, he is controlling opening and closing of the opens 
parts 50A and 50B of the shutter part 52, and is trying to open a gap or one side. 
[0049]Drawing ?■ is a figure explaining the photographing operation of the picture seen from 
two or more viewpoints depended on the imaging device concerning a 3rd embodiment of this 
invention. Drawing 7 (A) is a figure explaining photography of the picture seen from the opens 
part 50A, and dravving 7 (B) is a figure explaining photography of the picture seen from the 
opens part 50B. The photographic subject A whose distance parallel to an optic axis from the 
image formation part 14 for azimuth difference in the external world is the distance A here. 
There is the photographic subject C whose distance parallel to an optic axis is the distance C 
from the photographic subject B whose distance parallel to an optic axis from the image 
formation part 14 for azimuth difference is the distance B, and the image formation part 14 for 
azimuth difference, and it is assumed that the photographic subject A, B, and C is located in a 
line sequentially from the one where the value of the X-axis in a figure is larger. A distance 
parallel to an optic axis from the image formation part 14 for azimuth difference of the field of 
the distance B is a focusing point side of the image formation part 14 for azimuth difference. 
[0050]ln the parallax image input device body part 12, first, the control section 52 opens the 
opens part 50A of the shutter part 50, and closes the opens part 50B. Thereby, the picture of 
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the external world is connected via the opens part 50A on the acceptance surface of the image 
pick-up part 16 for azimuth difference. As shown in drawing 7 (A), the image of each 
photographic subject is connected to the image pick-up part 16 for azimuth difference in order 
of the photographic subject C, B, and A sequentially from the one where the value of the X- 
axis is larger, for example. The image pick-up part 16 for azimuth difference changes into 
image data the picture connected on the acceptance surface. 

[0051]Subsequently, the control section 52 opens the opens part SOB of the shutter part 50, 
and closes the opens part 50A. Thereby, the picture of the external world is connected via the 
opens part 50B on the acceptance surface of the image pick-up part 16 for azimuth difference. 
As shown in drawing 7 (B), the image of each photographic subject is connected to the image 
pick-up part 16 for azimuth difference in order of the photographic subject C, B, and A 
sequentially from the one where the value of the X-axis is larger, for example. As compared 
with the position of the image shown in drawing 7 (A), the position of the image shown in 
drawing 7 (B). The position of the image of the photographic subject B located on a focusing 
point side does not change, but the image of the photographic subject A located in the image 
formation part 14 side for azimuth difference from a focusing point side is connected to the 
position shifted in the direction to which the value of the X-axis becomes small, and the image 
of the photographic subject C located in the distance from a focusing point side is connected to 
the position shifted in the direction to which the value of the X-axis becomes large. The image 
pick-up part 16 for azimuth difference changes into image data the picture connected on the 
acceptance surface. 

[0052] Drawi ng 8 is a figure explaining logging operation of the picture concerning a 3rd 
embodiment of this invention, prawing 8 is a figure explaining logging of the picture connected 
via the opens part 50A or the opens part 50B as shown in drawing 7. The picture slicing part 
17 starts the 1st image data shown in drawing 8 (C) contained in the predetermined range to 
the image connected via the opens part 50A as shown in drawing 8 (A). Predetermined makes 
distance movement of said range carry out in the direction to which the value of the X-axis 
becomes small to the image connected via the opens part SOB, as shown in drawing 8 (B), and 
the picture slicing part 17 starts the 2nd image data shown in drawing 8 (D). 
[00S3]lf the 1st image data started in this way and the 2nd image data are piled up, the amount 
of gaps of the same photographic subject between both-images data will appear greatly, so 
that the distance of a photographic subject is far, as shown in drawing 8 (E). Thus, since it 
shifts, so that a photographic subject is far, and quantity appears greatly, depth can be 
specified only by calculating the relative amount of gaps of a photographic subject based on 
the image data seen from two or more viewpoints, and a throughput can be reduced. 
Supposing it calculates the amount of gaps of each photographic subject using dravying 8 (A) 
and (B) as compared with this, it will not be able to be specified in any before and behind a 
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focusing point side a piiotograpliic subject is, but it will be necessary to also recognize the 
direction which a gap generates, and, for this reason, processing will beconne connplicated only 
In the relative amount of gaps. The 1st innage data and the 2nd innage data which were started 
by doing in this way are memorized by the parallax picture storage parts store 18. About the 
processing after making the image data in the imaging device 10 memorize, since it is the 
same as that of a 1st above-mentioned embodiment, explanation is omitted here. 
[0054]This invention is not limited to the above-mentioned embodiment, and various 
modification is possible for It. For example, although the example was shown using CCD as 
the image pick-up part 26, this invention is not restricted to this, but Is provided with the 
installation section which installs the film as an example of a photochemical reaction member 
in the image pick-up part 26, for example, and it may be made to picturize a picture In the 
above-mentioned embodiment by the film installed in the installation section. In the above- 
mentioned embodiment, although explained taking the case of the digital camera, this 
Invention Is not restricted to this but can also be applied to other imaging devices, such as an 
endoscope which observes the inside of human being's abdominal cavity. 
[0055]Although the range which captures an image by moving the image pick-up part 16 for 
azimuth difference which changes the range of image data to start, or captures an Image was 
changed In the above-mentioned embodiment. This invention Is not restricted to this, but the 
range of the picture to start is changed and it may be made to change the range which 
captures an image by moving the image pick-up part for azimuth difference which captures an 
image. 

[0056]ln the above-mentioned embodiment, although It had the image formation part 14 for 
azimuth difference and the Image formation part 24 of another composition, this Invention Is 
not restricted to this but may have only a lens system with a single Image formation system. In 
this case, a single lens system may constitute the image formation part 14 for azimuth 
difference, and the Image formation part 24, and it may be made to make the picture of the 
external world connect to the Image pick-up part 16 for azimuth difference, and the Image pick- 
up part 26 by the light branching means of prism etc. A part of Image formation part 14 for 
azimuth difference and Image formation part 24 may be made common composition. Although 
the Imaging system had two or more image pick-up parts 16, i.e., Image pick-up part for 
azimuth difference, and Image pick-up parts 26, this Invention Is not restricted to this but may 
have only an Image pick-up part with a single imaging system. In this case, a single image 
pick-up part may function as the image pick-up part 16 for azimuth difference, and the image 
pick-up part 26. 

[0057]Although CCD was used as the Image pick-up part 16 for azimuth difference, this 
Invention Is not restricted to this, but Is provided with the Installation section which Installs a 
film In the Image pick-up part 16 for azimuth difference also as an example of a photochemical 
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reaction member, for example, and it may be made to picturize a picture in the above- 
mentioned embodiment by tliefilm installed in the installation section. If it does in this way, the 
picture from two or more viewpoints which developed the film and were incorporated into the 
film. For example, if the image data to incorporate is started according to the interval during the 
viewpoint at the time of capturing these images when incorporating as digital image data by a 
film scanner etc., processing at the time of detecting the depth position to a predetermined 
photographic subject [ image data / concerned ] can be made easy. 

[0058]As mentioned above, although this invention was explained using the embodiment, the 
technical scope of this invention is not limited to the range given in the above-mentioned 
embodiment. It is clear to a person skilled in the art that various change or improvement can 
be added to the above-mentioned embodiment. It is clear from the statement of a claim that 
the gestalt's which added such change or improvement it may be contained in the technical 
scope of this invention. 
[0059] 

[Effect of the lnvention]According to this invention, a parallax picture can be easily 
incorporated so that clearly from the above-mentioned explanation. The depth information of 
the photographic subject of the external world can be acquired easily. 



[Translation done.] 
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